Improving the Representation and Conversion of Mathematical Formulae by Considering their Textual Context

Mathematical formulae represent complex semantic information in a concise form. Especially in Science, Technology, Engineering, and Mathematics, mathematical formulae are crucial to communicate information, e.g., in scientific papers, and to perform computations using computer algebra systems. Enabling computers to access the information encoded in mathematical formulae requires machine-readable formats that can represent both the presentation and content, i.e., the semantics, of formulae. Exchanging such information between systems additionally requires conversion methods for mathematical representation formats. We analyze how the semantic enrichment of formulae improves the format conversion process and show that considering the textual context of formulae reduces the error rate of such conversions. Our main contributions are: (1) providing an openly available benchmark dataset for the mathematical format conversion task consisting of a newly created test collection, an extensive, manually curated gold standard and task-specific evaluation metrics; (2) performing a quantitative evaluation of state-of-the-art tools for mathematical format conversions; (3) presenting a new approach that considers the textual context of formulae to reduce the error rate for mathematical format conversions. Our benchmark dataset facilitates future research on mathematical format conversions as well as research on many problems in mathematical information retrieval. Because we annotated and linked all components of formulae, e.g., identifiers, operators and other entities, to Wikidata entries, the gold standard can, for instance, be used to train methods for formula concept discovery and recognition. Such methods can then be applied to improve mathematical information retrieval systems, e.g., for semantic formula search, recommendation of mathematical content, or detection of mathematical plagiarism.Comment: 10 pages, 4 figure

Mathematical Formulae in Wikimedia Projects 2020

This poster summarizes our contributions to Wikimedia's processing pipeline for mathematical formulae. We describe how we have supported the transition from rendering formulae as course-grained PNG images in 2001 to providing modern semantically enriched language-independent MathML formulae in 2020. Additionally, we describe our plans to improve the accessibility and discoverability of mathematical knowledge in Wikimedia projects further.Comment: Submitted to JCDL 2020: Proceedings of the ACM/ IEEE Joint Conference on Digital Libraries in 2020 (JCDL '20), August 1-5, 2020, Virtual Event, Chin

Too Late! Influence of Temporal Delay on the Neural Processing of One’s Own Incidental and Intentional Action-Induced Sounds

The influence of delayed auditory feedback on action evaluation and execution of real-life action-induced sounds apart from language and music is still poorly understood. Here, we examined how a temporal delay impacted the behavioral evaluation and neural representation of hurdling and tap-dancing actions in a functional magnetic resonance imaging (fMRI) experiment, postulating that effects of delay diverge between the two, as we create action-induced sounds intentionally in tap dancing, but incidentally in hurdling. Based on previous findings, we expected that conditions differ regarding the engagement of the supplementary motor area (SMA), posterior superior temporal gyrus (pSTG), and primary auditory cortex (A1). Participants were videotaped during a 9-week training of hurdling and tap dancing; in the fMRI scanner, they were presented with point-light videos of their own training videos, including the original or the slightly delayed sound, and had to evaluate how well they performed on each single trial. For the undelayed conditions, we replicated A1 attenuation and enhanced pSTG and SMA engagement for tap dancing (intentionally generated sounds) vs. hurdling (incidentally generated sounds). Delayed auditory feedback did not negatively influence behavioral rating scores in general. Blood-oxygen-level-dependent (BOLD) response transiently increased and then adapted to repeated presentation of point-light videos with delayed sound in pSTG. This region also showed a significantly stronger correlation with the SMA under delayed feedback. Notably, SMA activation increased more for delayed feedback in the tap-dancing condition, covarying with higher rating scores. Findings suggest that action evaluation is more strongly based on top–down predictions from SMA when sounds of intentional action are distorted

Neural changes when actions change: Adaptation of strong and weak expectations

Repeated experiences with an event create the expectation that subsequent events will expose an analog structure. These spontaneous expectations rely on an internal model of the event that results from learning. But what happens when events change? Do experience-based internal models get adapted instantaneously, or is model adaptation a function of the solidity of, i.e., familiarity with, the corresponding internal model? The present fMRI study investigated the effects of model solidity on model adaptation in an action observation paradigm. Subjects were made acquainted with a set of action movies that displayed an altered script when encountered again in the scanning session. We found model adaptation to result in an attenuation of the premotor-parietal network for action observation. Model solidity was found to modulate activation in the parahippocampal gyrus and the anterior cerebellar lobules, where increased solidity correlated with activity increase. Finally, the comparison between early and late stages of learning indicated an effect of model solidity on adaptation rate. This contrast revealed the involvement of a fronto-mesial network of Brodmann area 10 and the ACC in those states of learning that were signified by high model solidity, no matter if the memorized original or the altered action model was the more solid component. Findings suggest that the revision of an internal model is dependent on its familiarity. Unwarranted adaptations, but also perseverations may thus be prevented

VMEXT: A Visualization Tool for Mathematical Expression Trees

Mathematical expressions can be represented as a tree consisting of terminal symbols, such as identifiers or numbers (leaf nodes), and functions or operators (non-leaf nodes). Expression trees are an important mechanism for storing and processing mathematical expressions as well as the most frequently used visualization of the structure of mathematical expressions. Typically, researchers and practitioners manually visualize expression trees using general-purpose tools. This approach is laborious, redundant, and error-prone. Manual visualizations represent a user's notion of what the markup of an expression should be, but not necessarily what the actual markup is. This paper presents VMEXT - a free and open source tool to directly visualize expression trees from parallel MathML. VMEXT simultaneously visualizes the presentation elements and the semantic structure of mathematical expressions to enable users to quickly spot deficiencies in the Content MathML markup that does not affect the presentation of the expression. Identifying such discrepancies previously required reading the verbose and complex MathML markup. VMEXT also allows one to visualize similar and identical elements of two expressions. Visualizing expression similarity can support support developers in designing retrieval approaches and enable improved interaction concepts for users of mathematical information retrieval systems. We demonstrate VMEXT's visualizations in two web-based applications. The first application presents the visualizations alone. The second application shows a possible integration of the visualizations in systems for mathematical knowledge management and mathematical information retrieval. The application converts LaTeX input to parallel MathML, computes basic similarity measures for mathematical expressions, and visualizes the results using VMEXT.Comment: 15 pages, 4 figures, Intelligent Computer Mathematics - 10th International Conference CICM 2017, Edinburgh, UK, July 17-21, 2017, Proceeding

Carbon Oxidation State in Microbial Polar Lipids Suggests Adaptation to Hot Spring Temperature and Redox Gradients

The influence of oxidation-reduction (redox) potential on the expression of biomolecules is a topic of ongoing exploration in geobiology. In this study, we investigate the novel possibility that structures and compositions of lipids produced by microbial communities are sensitive to environmental redox conditions. We extracted lipids from microbial biomass collected along the thermal and redox gradients of four alkaline hot springs in Yellowstone National Park (YNP) and investigated patterns in the average oxidation state of carbon (ZC), a metric calculated from the chemical formulae of lipid structures. Carbon in intact polar lipids (IPLs) and their alkyl chains becomes more oxidized (higher ZC) with increasing distance from each of the four hot spring sources. This coincides with decreased water temperature and increased concentrations of oxidized inorganic solutes, such as dissolved oxygen, sulfate, and nitrate. Carbon in IPLs is most reduced (lowest ZC) in the hot, reduced conditions upstream, with abundance-weighted ZC values between −1.68 and −1.56. These values increase gradually downstream to around −1.36 to −1.33 in microbial communities living between 29.0 and 38.1◦C. This near-linear increase in ZC can be attributed to a shift from ether-linked to ester-linked alkyl chains, a decrease in average aliphatic carbons per chain (nC), an increase in average degree of unsaturation per chain (nUnsat), and increased cyclization in tetraether lipids. The ZC of lipid headgroups and backbones did not change significantly downstream. Expression of lipids with relatively reduced carbon under reduced conditions and oxidized lipids under oxidized conditions may indicate microbial adaptation across environmental gradients in temperature and electron donor/acceptor supply

Why you think Milan is larger than Modena: Neural correlates of the recognition heuristic

When ranking two alternatives by some criteria and only one of the alternatives is recognized, participants overwhelmingly adopt the strategy, termed the recognition heuristic (RH), of choosing the recognized alternative. Understanding the neural correlates underlying decisions that follow the RH could help determine whether people make judgments about the RH's applicability or simply choose the recognized alternative. We measured brain activity by using functional magnetic resonance imaging while participants indicated which of two cities they thought was larger (Experiment 1) or which city they recognized (Experiment 2). In Experiment 1, increased activation was observed within the anterior frontomedian cortex (aFMC), precuneus, and retrosplenial cortex when participants followed the RH compared to when they did not. Experiment 2 revealed that RH decisional processes cannot be reduced to recognition memory processes. As the aFMC has previously been associated with self-referential judgments, we conclude that RH decisional processes involve an assessment about the applicability of the RH

Surmising synchrony of sound and sight: Factors explaining variance of audiovisual integration in hurdling, tap dancing and drumming.

Auditory and visual percepts are integrated even when they are not perfectly temporally aligned with each other, especially when the visual signal precedes the auditory signal. This window of temporal integration for asynchronous audiovisual stimuli is relatively well examined in the case of speech, while other natural action-induced sounds have been widely neglected. Here, we studied the detection of audiovisual asynchrony in three different whole-body actions with natural action-induced sounds-hurdling, tap dancing and drumming. In Study 1, we examined whether audiovisual asynchrony detection, assessed by a simultaneity judgment task, differs as a function of sound production intentionality. Based on previous findings, we expected that auditory and visual signals should be integrated over a wider temporal window for actions creating sounds intentionally (tap dancing), compared to actions creating sounds incidentally (hurdling). While percentages of perceived synchrony differed in the expected way, we identified two further factors, namely high event density and low rhythmicity, to induce higher synchrony ratings as well. Therefore, we systematically varied event density and rhythmicity in Study 2, this time using drumming stimuli to exert full control over these variables, and the same simultaneity judgment tasks. Results suggest that high event density leads to a bias to integrate rather than segregate auditory and visual signals, even at relatively large asynchronies. Rhythmicity had a similar, albeit weaker effect, when event density was low. Our findings demonstrate that shorter asynchronies and visual-first asynchronies lead to higher synchrony ratings of whole-body action, pointing to clear parallels with audiovisual integration in speech perception. Overconfidence in the naturally expected, that is, synchrony of sound and sight, was stronger for intentional (vs. incidental) sound production and for movements with high (vs. low) rhythmicity, presumably because both encourage predictive processes. In contrast, high event density appears to increase synchronicity judgments simply because it makes the detection of audiovisual asynchrony more difficult. More studies using real-life audiovisual stimuli with varying event densities and rhythmicities are needed to fully uncover the general mechanisms of audiovisual integration

Impairment of Auditory-Motor Timing and Compensatory Reorganization after Ventral Premotor Cortex Stimulation

Integrating auditory and motor information often requires precise timing as in speech and music. In humans, the position of the ventral premotor cortex (PMv) in the dorsal auditory stream renders this area a node for auditory-motor integration. Yet, it remains unknown whether the PMv is critical for auditory-motor timing and which activity increases help to preserve task performance following its disruption. 16 healthy volunteers participated in two sessions with fMRI measured at baseline and following rTMS (rTMS) of either the left PMv or a control region. Subjects synchronized left or right finger tapping to sub-second beat rates of auditory rhythms in the experimental task, and produced self-paced tapping during spectrally matched auditory stimuli in the control task. Left PMv rTMS impaired auditory-motor synchronization accuracy in the first sub-block following stimulation (p<0.01, Bonferroni corrected), but spared motor timing and attention to task. Task-related activity increased in the homologue right PMv, but did not predict the behavioral effect of rTMS. In contrast, anterior midline cerebellum revealed most pronounced activity increase in less impaired subjects. The present findings suggest a critical role of the left PMv in feed-forward computations enabling accurate auditory-motor timing, which can be compensated by activity modulations in the cerebellum, but not in the homologue region contralateral to stimulation